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1N THE CLAIMS 

Page 21, change "Patent Claims" to —What is claimed is--. 
Cancel claims 1-20 and add new claims 21-71, reading as follows: 

—2 1 . (New) A method for increasing the depth discrimination of optically 
imaging systems comprising the steps of: 

projecting a periodic structure in the object; 

detecting N, N>=3 structured images of the object with phase angles 

2tt 

<p n = n ~^~ > n -0..N -I of the projected structure; and 

generating optical sections by evaluation of the images with the N different 
phase angles by Equation 5b. 

22. (New) A method for increasing the depth discrimination of optically 
imaging systems comprising the steps of: 

projecting a periodic structure in the object; 

detecting three structured images of the object with the three phase angles 0°, 
120°, 240° of the projected structure; and 

generating optical sections by evaluation of the three images with different 
phases by Equation 10. 

23 . (N ew) A method for increasing the depth discrimination of optically 
imaging systems comprising the steps of: 

projecting a periodic structure in the object; 

detecting four structured images of the object with the four phase angles 0°, 
90°, 180°, 270° of the projected structure; and 

generating optical sections by evaluation of the four images with different 
phases by Equation 1 1 . 



24. (New) A method for increasing the depth discrimination of optically 
imaging systems comprising the steps of: 

projecting a periodic structure in the object; 



detecting N, N>=3 structured images of the object with phase angles 

<p n = n — , n = 0..N - 1 of the projected structure; and 

calculating structure- free images by Equation 7, wherein the image of the 
object calculated in this way does not have increased depth discrimination. 

25. (New) A method for increasing the depth discrimination of optically 
imaging systems comprising the steps of: 

projecting a periodic structure in the object; 

detecting two structured images of the object with phase angles of 0° and 

180°; and 

calculating a synthetic image by Equation 12c, wherein the image of the object 
calculated in this way does not have increased depth discrimination. 

26. (New) A method for increasing the depth discrimination of optically 
imaging systems comprising the steps of: 

projecting aperiodic structure in the object; 

detecting four structured images of the object with phase angles of 0°, 90°, 
180°, 270°; and 

calculating a synthetic image by Equation 7, wherein the image of the object 
calculated in this way does not have increased depth discrimination. 

27. (New) A method for increasing the depth discrimination of optically 
imaging systems comprising the steps of: 

projecting a periodic structure in the object; 

detecting three structured images of the object with phase angles of 0°, 120° 

and 240°; and 

calculating a synthetic image by Equation 12a, wherein the image of the object 
calculated in this way does not have increased depth discrimination. 



28. (New) A method for increasing the depth discrimination of optically 
imaging systems comprising the steps of: 

projecting a periodic structure in the object; 



detecting four structured images of the object with phase angles of 0°, 90°, 
180°, 270°; and 

calculating a synthetic image by Equation 12b, wherein the image of the 
object calculated in this way does not have increased depth discrimination. 

29. (New) The method of claim 21 including using said method in all 
linear interactions. 

30. (New) The method of claim 21 including using said method in 

microscopy. 

3 1 . (New) The method of claim 2 1 including using said method in 
incident light microscopy. 

32. (New) The method of claim 21 including use in incident brightfield 

microscopy. 

33. (New) The method of claim 21 including use in transmitted light 

microscopy. 

34. (New) The method of claim 21 including use in incident 
fluorescence microscopy. 

35. (New) An arrangement for increasing the depth discrimination of 
optically imaging systems comprising: 

means for projecting a periodic structure in the object; 

means for detecting N, N>=3 structured images of the object with phase 

271 

angles cp n =n — , n = 0..N - 1 of the projected structure; and 
N 

means for generating optical sections by evaluation of the images with the N 
different phase angles by Equation 5b. 
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36. (New) An arrangement for increasing the depth discrimination of 
optically imaging systems comprising: 

means for projecting a periodic structure in the object; 

means for detecting three structured images of the object with the three phase 
angles 0°, 120°, 240° of the projected structure; and 

means for generating optical sections by evaluation of the three images with 
different phases by Equation 10. 

37. (New) An arrangement for increasing the depth discrimination of 
optically imaging systems comprising: 

means for projecting a periodic structure in the object; 

means for detecting four structured images of the object with the four phase 
angles 0°, 90°, 180°, 270° of the projected structure; and 

means for generating optical sections by evaluation of the four images with 
different phases by Equation 1 1 . 

38. (New) An arrangement for increasing the depth discrimination of 
optically imaging systems comprising: 

means for projecting a periodic structure in the object; 

means for detecting N, N>=3 structured images of the object with phase 

2n 

angles (p n - n— , n = 0.JV-1 of the projected structure; and 

means for calculating structure- free images by Equation 7, wherein the image 
of the object calculated in this way does not have increased depth discrimination. 

39. (New) An arrangement for increasing the depth discrimination of 
optically imaging systems comprising: 

means for projecting a periodic structure in the object; 
means for detecting two structured images of the object with phase angles of 
0° and 180°; and 

means for calculating a synthetic image by Equation 12c, wherein the image 
of the object calculated in this way does not have increased depth discrimination. 
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40. (New) An arrangement for increasing the depth discrimination of 
optically imaging systems comprising: 

means for projecting a periodic structure in the object; 

means for detecting four structured images of the object with phase angles of 
0°, 90°, 180°, 270°; and 

means for calculating a synthetic image by Equation 7, wherein the image of 
the object calculated in this way does not have increased depth discrimination. 

41 . (New) An arrangement for increasing the depth discrimination of 
optically imaging systems comprising: 

means for projecting a periodic structure in the object; 

means for detecting three structured images of the object with phase angles of 
0°, 120° and 240°; and 

means for calculating a synthetic image by Equation 12a, wherein the image 
of the object calculated in this way does not have increased depth discrimination. 

42. (New) An arrangement for increasing the depth discrimination of 
optically imaging systems comprising: 

means for projecting a periodic structure in the object; 

means for detecting four structured images of the object with phase angles of 
0°, 90°, 180°, 270°; and 

means for calculating a synthetic image by Equation 12b, wherein the image 
of the object calculated in this way does not have increased depth discrimination. 

43. (New) The arrangement for increasing the depth discrimination of 
optically imaging systems according to claim 35, wherein the spatial phase of the projected 
structure is adjusted by a plane-parallel plate which is rotatable about an axis perpendicular to 
the optical axis. 

44. (New) The arrangement according to claim 35, wherein the spatial 
phase is adjusted by a galvanometer scanner. 
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45. (New) The arrangement according to claim 35, wherein the structure 
can be move in axial direction in addition. 

46. (New) The arrangement according to claim 35, wherein the 
illumination-side tube lens can be moved in axial direction in addition. 

47. (New). The arrangement according to claim 45 with motor-actuated 

movement. 

48. (New). The arrangement according to claim 46 with motor-actuated 

movement. 

49. (New) The arrangement according to claim 47 with motor-actuated 
movement corresponding to Figures 7a, 7b. 

50. (New) The arrangement according to claim 47 with motor-actuated 
movement of an optical wedge corresponding to Figure 9. 

5 1 . (New) The arrangement according to claim 35 in combination with 
the measurement of the light intensity by a light-sensitive detector. 

52. (New) The arrangement according to claim 51 in combination with 
the measurement of the light intensity by a light-sensitive detector, wherein a photodiode is 
used as light-sensitive detector. 

53. (New). The arrangement according to claim 51, wherein the digitized 
signal of the light-sensitive detector is used for scaling the image brightness. 



54. (New) The arrangement according to claim 53, wherein the digitized 
signal of the light-sensitive detector is used for scaling the image brightness using Equation 
17. 




55. (New) The arrangement according to claim 35, wherein the calculation 
of depth-discriminated images is obtained by solving the system of equations given by 
Equations 20, 21 and 22. 



56. (New) The arrangement according to claim 35 also including use 
of an automatic shutter. 

57. (New) The arrangement according to claim 35 also including use of a 
shutter according to Figure 5. 

58. (New) The arrangement according to claim 35 also including 
minimizing artifacts through the use of averaging according to Equation 24. 

59. (New) The arrangement according to claim 35 also including 
implementation of a module with two positions according to Figure 8. 

60. (New) The arrangement according to claim 35 also including the 
coding of the grating with a bar code for automatic detection of the grating. 

61 . (New) The arrangement according to claim 35 also including the 
coding of the grating with a stripe code for automatic detection of the grating. 

62. (New) The arrangement according to claim 35 also including 
exchangeable gratings. 

63. (New) The arrangement according to claim 62 wherein the illumination- 
side tube lens is moved in axial direction in addition. 



64. (New) The arrangement according to claim 63 with motor-actuated 

movement. 



65. (New) The arrangement according to claim 63 with motor-actuated 
movement corresponding to Figures 7a, 7b. 



66. (New) The arrangement according to claim 63 with motor-actuated 
movement of an optical wedge corresponding to Figure 9. 

67. (New) The arrangement according to claim 63 in combination with the 
measurement of the light intensity by a light-sensitive detector. 

68. (New) The arrangement according to claim 67 in combination with the 
measurement of the light intensity by a light-sensitive detector, wherein a photodiode is used 
as light-sensitive detector. 

69. (New) The arrangement according to claim 67, wherein the digitized 
signal of the light-sensitive detector is used for scaling the image brightness. 

70. (New) The arrangement according to claim 69, wherein the digitized 
signal of the light-sensitive detector is used for scaling the image brightness using Equation 
17. 

71. (New). The arrangement according to claim 70, wherein the calculation 
of depth-discriminated images is obtained by solving the system of equations given by 
Equations 20, 21 and 22.— 

IN THE ABSTRACT 

Cancel the present Abstract and substitute therefor the enclosed Abstract which is 
attached to the Substitute Specification. 

REMARKS 

Claims 1-20 have been cancelled and new claims 21-71 have been added. The 
amendments to the claims have been made only to improve the form of the claims for 
examination purposes. 
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The specification and abstract have been amended to conform it to U.S. 
An early and favorable action on the merits is respectfully requested. 



August 12, 2002 
REED SMITH LLP 
599 Lexington Avenue 
New York, NY 10022-7650 

GHK:jl 

Enc: Substitute Specification 
Abstract 
Marked-up/Bolded Versioins 



Respectfully submitted, 




'Gferald H. Kiel 
Reg. No. 25,116 
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BACKGROUND OF THE INVENTION 

a) Field of the Invention 
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5 [5] Ben Levy, et al. 

PCT Application WO 97/06509, "Imaging measurement system" 
Priority date: August 3, 1995 
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10 

[6] T. Wilson, et al. 
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conventional microscope", Optics Letters 22 (24) (1997) 

15 [7] T. Wilson, et al. 

"Real-time three-dimensional imaging of macroscopic structures", 
Journal of Microscopy 191 (2) (1998) 

[8] M. A. A. Neil, et al. 

20 "Real time 3D fluorescence microscopy by two beam interference 

illumination", Optics Communications 153, (1998) 

[9] F. Lanni, et al. 

"Grating Image Systems for Optical Sectioning Fluorescence 
25 Microscopy of Cells, Tissues and Small Organisms", published in 

Imaging Neurons - A Laboratory Manual, edited by R. Yuste, F. 
Lanni, A, Konnerth, Cold Spring Harbor Laboratory Press (2000) 

[10] Q. S. Hanley, P. J. Verveer, T. M. Jovin 

30 "Optical sectioning fluorescence spectroscopy in a programmable 

array microscope", Applied Spectroscopy 52 (6): 783 (1998) 



BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows a simplified optical schematic of structured 
illumination. The image-forming beam path is shown. A one-dimensional 
transmission grating (3) located in a focus plane of the optical arrangement shown in 
the drawing is illuminated by a light source (1) and collector optics (2) succeeding 
the latter. The transmission grating can be moved in axial direction (parallel to the 
optical axis of the system). Possible movements are indicated by the arrow shown in 
the drawing. The grating is succeeded in the direction of light by a plane-parallel 
glass plate (4). The angle of the plane-parallel plate with respect to the optical axis 
can be adjusted in a defined manner. The structure is imaged in the specimen plane 
(7) by the succeeding illumination-side lensesl (5 and 6) (tube lens and objective or 
condenser). The possible movements of the lens (5) (tube lens) indicated by an 
arrow can be used as an alternative solution to the movement of the grating (3), since 
an axial displacement of the tube lens substantially equals an axial displacement of 
the focus plane. 

Light proceeding from the specimen is imaged again through a pair of 
lenses (8 and 9) (objective and tube lens) in the focus plane (10) following the latter. 

Figure 2 schematically shows the longitudinal chromatic aberrations 
which occur, in general, in the optical arrangement from Figure 1 . The image- 
forming beam paths for two different wavelengths are shown. Due to the 
longitudinal chromatic aberration of the lens combinations, the focus planes for 
different wavelengths extend axially at different locations. This longitudinal 
chromatic aberration can be corrected by an axial displacement of the grating (or of 
the tube lens (5)). This is indicated by the grating positions (3a) and (3b). Further, 
the same axial location in the specimen is not sharply imaged for both wavelengths. 
This is shown by the focus planes (7a) and (7b). The difference in the position of the 
focus planes (7a) and (7b) leads, e.g., with multifluorescence, to an erroneous 
acquisition of the object. The chromatic shift of the focus planes in the object can be 
taken into account by a software correction of the axial positions. 

Figure 3 shows a construction of the optical arrangement from Figure 
1 or Figure 2 for incident fluorescence detection in microscopy. A structure 3 (sine 



1 By "lenses" is meant, in general, also complex lens systems. 
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grating or binary transmission grating) is illuminated by a lamp (1) and succeeding 
lens (2). The position of the structure can be varied axially, e.g., by a motor-driven 
eccentric movement (see Figure 7). For control purposes, the motor of the eccentric 
is connected to the control electronics by a cable. A plane-parallel transparent glass 
5 pl^te (4) (e.g., made from BK7 or fused silica) succeeding the latter can be offset in 

different angular positions relative to the optical axis, e.g., by means of a scanner 
(Figure 7). For control purposes, the scanner is connected to the electronics box by a 
cable. The spatial phase of the structure can be varied (Figure 5) by the parallel 
offset of the plate. The succeeding (illumination-side) tube lens (5) images the 

10 structure in the infinity space of the ICS optics of the microscope. The structure is 

imaged in the specimen plane through the excitation filter (7), the dichroic beam 
splitter (11) and the objective (10). The preparation or specimen (9) is located on 
the vertically adjustable object stage (8). The height of the stage (8) can be adjusted 
in the direction indicated by the arrow by the drive mechanism (6). For this purpose, 

15 the drive mechanism is connected to the electronics control unit (15) by a cable. The 

fluorescent light emitted by the specimen is separated from the excitation light by 
the dichroic color splitter and the emission filter (12) succeeding the latter. The 
excitation filter, dichroic beam splitter and emission filter are grouped together (18). 
Finally, the structured fluorescence emission is focused on the intermediate image 

20 plane of the camera by a tube lens (13). The CCD chip of the camera (14) detects 

the fluorescence emission which is structured in this way. The camera data are 
transferred to the PC (16). The axial position of the structure (grating) and the angle 
of the plane-parallel plate relative to the optical axis can be adjusted by the PC via 
the electronic interface with the control electronics. For this purpose, the control 

25 electronics are connected to a scanner, e.g., a galvanometer scanner (e.g., M2T, 

manufactured by General Scanning, 500 Arsenal Street, Watertown, MA 02472). 
Finally, the spatial phase of the grating can accordingly be adjusted (Figure 6). 
Further, the axial position of the grating is adjustable. In this way, longitudinal 
chromatic aberrations can be minimized (Figure 2). This enables the spatial-phase- 

30 dependent acquisition and calculation of the structured images for calculating an 

optical section image under different chromatic conditions. The result of the 
calculation is displayed on the monitor 17. In order to measure the illumination 
intensity, a small proportion of the excitation light is coupled out, e.g., via a plane- 
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parallel plate (20), and focused on a photodiode through a lens. The voltage of the 
photodiode is supplied to the control electronics via a cable connection. After 
analog-digital conversion, the digital value is used for correcting instabilities of the 
light source. 

5 Possible implementations of the axial displacement of the grating and 

adjustment of the angle of the plane-parallel plate are shown in Figure 7. 

Figure 4 shows a corresponding arrangement for a reflecting 
specimen. For this purpose, the component group (18) shown in Figure 3 is replaced 
by a 50:50 beam splitter (21) (e.g., a semitransparent mirror). 

10 Figure 5 shows an expanded system with an electronic shutter for 

controlling the exposure cycles for camera detection. The shutter shown here 
comprises a plane mirror (22) whose angle relative to the optical axis can be 
adjusted, e.g., by a galvanometer scanner, followed by a prism (21), and a stationary 
mirror (23). Through the combination of the plane mirror whose angle relative to 

15 the optical axis is adjustable, the prism and the fixed plane mirror, an "open" 

position (beam path indicated by solid lines) and a "closed" position of the optical 
arrangement (beam path in dashed lines) can be realized. 

The arrangement makes it possible to switch the light source on and 
off rapidly. Realistic switching times are in the range of about 1 ms to 10 ms. The 

20 minimum shutter times are limited by the response times of the scanner with typical 

mirror diameters of about 20 mm. 

The essential advantage in this case consists in the fast switching time 
of the arrangement and the negligible wear of the described arrangement compared 
to normal wear. 

25 Figure 6 is a schematic diagram showing the adjustment of the spatial 

phase of the grating. The square grating 1 shown in the drawing is imaged via a 
plane-parallel glass plate 2. The image of the grating 3 can be displaced depending 
on the angle of the plate. This makes it possible to adjust the phase of the grating 
without mechanical movement thereof. 

30 Figure 7a shows a side view of the mimic for adjusting the axial 

position of the grating and the adjustment of the angle of the plane-parallel plate. 
The axial position (height) of the structure (2) can be adjusted by means of a motor 
(4)-controlled eccentric (3). A galvanometer scanner (5) makes it possible to adjust 



the angle of the plane-parallel plate (1) by a rotating movement. The same 
arrangement is shown in a top view in Figure 4b. 

Finally, Figure 8 shows a construction of the overall module in which 
two diaphragm apertures are provided. The aperture designated as position 1 is a 
5 structure- free optical passage with a variable iris diaphragm (field diaphragm of the 

microscope). Position 2 contains the optical arrangement, which was described in 
detail above, comprising structure and plane-parallel plate in the optical beam path. 
The arrangement shown in this Figure makes it possible to change, for example, 
from conventional incident light microscopy to structured illumination in a simple 
10 manner. For this purpose, the entire module shown here must be moved 

perpendicular to the optical axis into the desired position. 

Figure 9 shows a possibility for correcting the longitudinal chromatic 
aberration in the LFB plane. For this purpose, two optical glass wedges (1) (2) are 
used. One of the two wedges can be displaced, e.g., by a linear motor, in the 
15 direction indicated by the arrow. The arrangement shown in the drawing allows the 

optical path length to be varied. 

In Figure 10, the changing of the optical path length for correcting the 
longitudinal chromatic error by means of optical wedges (Figure 9) is integrated in a 
construction of the optical system. This Figure corresponds to Figure 3 with the 
20 exception of the wedge arrangement (21). 

Figure 1 1 is a schematic view of the microscope system and the 
electronic components. 

Figure 12 shows the carrier of the grating from Figure 7 (2) in an 
embodiment with coding. The five electronic contacts shown in the drawing are 
25 used by the conducting pins, also shown, to realize the desired binary information 

for automatic detection of the implemented grating carriers. Coded gratings are 
useful, particularly when gratings are exchanged mechanically, so as to prevent user 
errors due, for example, to incorrectly adjusted spatial phases for calculating section 
images by the automatic detection. 
30 The automatic detection of a mechanical module is implemented in 

Figure 13. A photosensitive receiver can be covered or opened by the relative 
movement, indicated by the arrow, between the two interfaces or boundary surfaces. 
Automatic position detection can be realized in this way. 
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DESCRIPTION OF THE METHOD AND PRFERRED EMBODIMENTS 

The principle of structured illumination will be described in the 
following. A structure is located in the plane of the field diaphragm (LFB) of a 
5 microscope. In the following, the structure is a one-dimensional or two-dimensional 

periodic structure. For example, the one-dimensional or two-dimensional period 
structure (grating) can be a transmission grating or reflection grating fabricated by 
lithography. The arrangement of the grating can be binary (square grating) or 
sinusoidal (sine grating). The grating is imaged in the object plane by suitable 

10 optical elements (illumination-side tube lens and, e.g., objective). A reflecting or 

fluorescing object, for example, is located in the object plane. The lateral structure 
of the illumination intensity, with linear interaction, results in a structured object. 
The structured object is imaged, in turn, via the objective and the TV-side tube lens, 
in the succeeding intermediate image plane and is registered by an array detector (see 

15 Figure). The sensor surface of the detector is situated in the intermediate image 

plane (TV output at the microscope). 

The increase in depth contrast in a microscope recording with 
structured illumination is ultimately attributable to the fact that only the mean value 
(or DC component) in conventional microscope imaging does not disappear when 

20 defocusing. However, when an object with a structure whose spatial frequency (or 

frequencies) is (or are) not equal to zero is illuminated and only the proportion of 
image information that was imaged in a structured manner is subsequently evaluated, 
the DC component (zero frequency) can be numerically removed. This increases the 
depth contrast. The structure of the illumination "marks" the portion of the object 

25 located in the focus of the optical system. 

Essentially the modulation depth (AC component) of the structured 
object must be determined by pixel in order to calculate the optical section. In the 
simplest case of a one-dimensional periodic sinusoidal structure, with a linear 
interaction, the image-side intensity I— I(x,y,(p) with spatial variables x,y and the 

30 parameter for the spatial phase angle of the periodic structure (p is given by 



I(x, y, <p) = a 0 Xt y + m ■ sin(* t • x + cp ) 



Equation 1 
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where the locally dependent modulation depth m xy =:m(x t y). 

On the one hand, the modulation depth is dependent on the grating 
frequency, the imaging optics and the object being viewed. 
5 Using addition theorems, this can be written as 

t x .y(9>) = *ox.y + a n,y'M n (<P) + b ix iy -cos(<p) Equation 2 

where (p is the phase angle of the sinusoidal structured illumination, a 0Xiy is the DC 
10 component (= mean value) and 

a Xxy = m xy co$(k x x) Equation 3 

b Xxy = m xy s\n{k x x) Equation 4 

15 

so that it immediately follows for the local modulation depth of the sine frequency: 



m Xt} , = -! oc ^a ljcj , + b Xxy . Equation 5a 



a 0x,y 



20 An optical section image is precisely the modulation depth multiplied by the image 

intensity (= DC component) 



cy'= m x, y I = ^a Xx / +b Xx>y 2 Equation 5b 



' sec tioned xy ™x. 



25 In addition, the phase angle can be calculated 



tan(<p) = ^ 



Equation 6 



-9- 



The coefficients a/ A> ,and b/ xy , which are dependent on the modulation 
frequency, and the locally dependent mean value can initially be written as follows 
by changing the phase of the discrete Fourier expansion: 

2n 



Ox,y 



1 N-l 

i = -Vf 



*,y\ N 



Equation 7 



2 



2n 



( 2n^ 
n — 



Equation 8 



. 2 $H ( 2n\ ( In 



N N 



Equation 9 



10 



15 



where N is the quantity of phase steps. 

By inserting Equations 7 - 9 in Equation 5, the local modulation 
depth m xy can be calculated. 

For three or four phase steps, the formulas are simplified for the last 
optical sections sought: 



sectioned xy ™x,y 



V2 
3 





r 2*Y 


2 






v. 3 J. 


+ 





An 
3 J 



2 r 



l x,y 



2n\ 4n 



-,2 



Equation 10 



20 for three phase steps (0, 2n/3 3 4rc/3), and 



1 

/ _ - m — - 

sectioned^ x,y 2 



-1 2 



/ (0) - / (?t) 

. x,y x,y y \ 



1 f 3 ^ 

x,yy2 J x,yy 2 



~|2 



Equation 1 1 



25 for four phase steps (0, tc/2, k, 3rc/2). 
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In order to calculate the DC component, the "mean value" must be 
calculated over the phase images (Equation 7). For three phase angles: 



Ix fV - a Qxy - — 



2/r 



v j J 



and, in a corresponding manner, for four phase angles: 



Equation 12a 



Ix, y - a Qxy - 



2 j + / ^ ( ^ ) + / *.^Y 



Equation 12b 
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steps: 



The simplest case is the calculation of the mean value with two phase 



1 = <V, - ~ (l Xty (0) + I x y (7i )) 



Equation 12c 



Because of the displacement set for the trigonometric functions, the 

15 modulation frequencies are eliminated in the calculation with the help of the last 

equations 12c. This simple algorithm is especially advantageous also when using a 
square grating (see below) because all higher harmonics of the square grating are 
suppressed. For 7^= 2", where n is a natural number, this is always the case. In 
general, when more than two phase steps are used, this is not the case. For example, 

20 with three phase steps (Equation 12a) the third harmonic leads to stripe artifacts 

when the mean value is formed. 

The DC component calculated generally by Equation 7 and Equation 
12 corresponds to conventional recording in the microscope. The depth contrast is 
not increased in this case. The calculation of the brightfield information with 

25 Equation 7 (12) is particularly applicable in an advantageous manner, since this 

processing makes it possible to switch from optical section to conventional 
recording without having to mechanically remove the structure from the beam path. 

When both the DC component and an optical section (AC 
component) are calculated, the out-of-focus information can also be determined 

30 (after suitable calibration) by subtraction (DC-AC). 
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When using a square grating, the following series expansion can be 
used for illustration: 



I(x,y,<p) = a 0xy + ]T- 



fc r (2/ + l) 



9 



Equation 13 



m> (2i + 1) 

The modulation depth of the grating frequency when focusing, e.g., 
on a mirror as specimen, can be described by the modulation transfer function [T. 
Wilson]. 



a cos 




\2NA) IN A 



Equation 14 



10 with incoherent illumination and incident illumination, where 

NA = numerical aperture 
X = wavelength 
k = spatial frequency. 

The formula above is applicable for circular pupils. In defocusing, a wave 
1 5 aberration, as complex phase factor, must be multiplied by the pupil function. 

Referring to Equation 10, it will be seen that the three-phase formula 
implicitly suppresses the third harmonic of the Fourier expansion of a square grating 
(Equation 13, i=l). Not until the fifth harmonic (i=2) does an interference term and, 
therefore, a residual modulation (artifact) result. When the one-dimensional grating 
20 is so dimensioned that the fifth harmonic can not be transmitted through the optical 

lowpass, i.e., is selected so as to be greater than the optical limit frequency 



Km* = "y~' Equation 15 

the artifacts can be completely suppressed by higher harmonics. This is not the case 
25 when Equation 1 1 is applied, i.e., when using four phase steps for calculating the 

(local) modulation depth. Therefore, Equation 1 1 can be used in connection with 
square gratings only to a limited extent, e.g., in case even the third harmonic in the 
Fourier expansion is not transmitted through the optics. It is advantageous with 
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respect to Equation 1 1 , under this additional pre-condition, that the second harmonic 
of the grating can be implicitly suppressed by the algorithm. 

Technical Implementation 

Technical implementations are shown in Figures 3 and 4 and further 
details on the implementations are shown in Figures 5 to 8. 

The following details of the implementations which are to be applied 
in the imaging systems for application in fluorescence microscopy (Figure 3) and 
reflection microscopy (Figure 4) will once again be presented in this connection. 
The relevant implementations are: 

Phase adjustment by means of a plane-parallel plate (plane- 
plate micrometer) 

The (motor-actuated) movement of the grating in axial 
direction for correcting longitudinal chromatic errors (focus correction) 

Correction of instability in the light source through the 
measurement of the light intensity by means of a photodetector (typically a 
photodiode) and subsequent numerical correction by scaling the acquired individual 
phase images 

Correction, e.g., of bleaching of dyes in fluorescence 
microscopy by means of an optimized averaging process 

Implementation of a fast shutter by means of scanners 
corresponding to Figure 5 

Correction, e.g., of statistical changes in the spatial phase of 
the grating by solving a system of equations 

Implementation of a system for the use of a two-dimensional 
grating using two orthogonal plane-plate micrometers. 

Phase displacement by means of a plane-parallel plate 

The adjustment of the phase position of the grating can be 
advantageously realized by means of parallel displacement (plane-plate micrometer). 
The principle is shown in Figure 6. A plane-parallel glass plate (e.g., made of BK7) 
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is arranged after the grating and, e.g., well-defined angles are adjusted by means of a 
galvanometer scanner. The use of a scanner offers the advantage in particular of a 
precise and fast adjustment of the angle. Typical adjusting angles in the range of 
about +/- 15° with an adjusting accuracy of <50 (irad and an adjusting time of less 
than 5 ms are technically possible. 

The data can be recorded synchronous with the detector in two ways: 
by a continuous movement of the plane-parallel plate and 
synchronization with the camera 

by a stepwise, sequential approach to the scan position 

Sequential positioning offers the advantage of a higher modulation 
depth (= contrast) of the structured object and independence of the contrast from the 
ratio of exposure time to readout time of the camera. 

In order to optimize the excitation-side transmission on the one hand 
and to minimize multiple reflections at the plane-parallel plate on the other hand, it 
is advantageous to minimize the reflections at all optical components, particularly 
the structure (or the substrate) and the plane-parallel plate, by antireflection coatings. 
Further, it is advantageous to insert the plane-parallel plate into the optical beam 
path at a small angle (not equal to 0°). Interfering reflections can be eliminated in 
this way. 

Focus correction, compensation of longitudinal chromatic aberration 

Although the imaging, for example, of fluorescing objects and 
reflecting objects, is identical in some respects, the differences between excitation 
wavelength and emission wavelength (Stokes shift), e.g., in fluorescing objects, 
present several essential points to be considered. 

Since, as a rule, chromatically corrected objectives also have a 
longitudinal chromatic aberration, the grating, object and TV intermediate image 
plane are each located in the optically conjugate planes only for one 
excitation/emission wavelength. With a change in the spectral ratios (filter change) 
or when the objectives are exchanged, the focus position of the object, grating and 
camera that has already been adjusted is no longer given. This leads to a reduction 
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in contrast in the structured image and, with larger longitudinal chromatic 
aberrations, to stripe artifacts in the calculated optical section image. 

The focus adjustment of the object on the intermediate image plane of 
the camera can be realized by refocusing. However, it is also necessary to correct 
the grating position relative to the object plane. The advantageous use of a manual 
or motor-operated adjusting possibility for focus adjustment of the grating on the 
object is shown in the present patent application. This can be implemented in two 
different ways using ICS optics: 

a) Displacement of the tube lens in Figure 1 . 

b) Displacement of the grating position in Figure 2. 
Even when the wavelengths for excitation and detection are 

substantially identical (reflection microscopy), an axial movement of the structure or 
illumination-side tube lens may also be necessary for correction due to chromatic 
aberrations. In particular, this is the case when changing from the visible spectral 
range to UV, for example. 

The grating position can be adapted, for example, by an eccentric 
movement (Figures 7a, 7b). Given sufficient space (or a sufficiently small necessary 
focus correction), it is possible to use an optical wedge and, therefore, to change the 
optical path length (Figures 9, 10). 

Correction of time-dependent intensity instability of the light source 

A change over time in the intensity of the light source during the 
detection of the phase images, generally N>^3 phase images, leads to stripe artifacts 
in the calculated optical section image, since a light source that is constant over time 
is assumed in the derivation of the formulas given above. The following steps can 
be taken to eliminate this influence of time-dependent changes in illumination which 
interferes with image quality. In a first step, a measurement value for the 
illumination intensity (shown in Figures 3, 4, 5) is obtained for the intensity of the 
light source, e.g., by imaging on a light-sensitive receiver, e.g., a commercially 
available photodiode. The time interval for the integration must be matched to the 
time interval for the exposure of the CCD chip of the camera. The voltage signal of 
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the photodiode circuit is subsequently digitized by means of an A-D converter. 
After measurement of the N individual images with the spatial phases 



<P n - •> Equation 1 6 

they can be scaled, e.g., as follows: 

I norm K v (<Pn ) = ~~" Equation 1 7 

with the digitized photodiode signal 



PD signal := — — 1 Equation 1 8 



averaged over the image phases, and the correction factor 



<- ) — z^r Equation 1 9 

"^Signal 



Phase instability, phase adjustment speed 

In order to minimize phase instability, it is advantageous to use a 
temperature-stabilized galvanometer scanner (e.g., M2T, manufactured by General 
Scanning). A scanner of this type also enables a short phase adjustment time of a 
few milliseconds. In order to optimize phase stability, the scanner is controlled 
electronically by a closed control loop. 

In addition, an SW correction of phase instabilities can be carried out 
by measuring the phases and subsequently approximating with a sine function 
according to the method described in the following. 

While an exact displacement of the projected structure by an Nth of 
the period is assumed in Equations 7-9, the modulation depth can also be 
determined when the intensity is recorded in three or more virtually optional 
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positions. First, the phase angle (p in Equation 1 must be measured for all positions 
of the structure. This can be carried out in different ways: 

1 . By measurement at the mechanical actuating element 
responsible for the displacement of the structure (e.g., the scanner) 

2. By determining from the recorded image, e.g., by Fourier 
transformation or by approximation of the recorded intensity values with a sine 
function 

3. Determining as in 2) above, but with a sensor which directly 
records the projected structure. This can also be achieved in that an optical element 
which images the structure directly on a partial area of the detector that is also used 
for recording the intensities of the specimen is arranged in the beam path. 

When determining the displacement of the structure, it is sufficient to 
determine the displacement relative to the first position. 

To determine the modulation depth, N intensity values which were 
recorded at N positions of the structure are considered and the index n is given in 
Equation 2 for the nth position: 



Equation 20 



The measured intensities M xy {<p n ) can deviate from the expected 
intensities I x ,y(<Pn)- The method of least error squares can be used for approximation: 



AM . . 



Equation 21 



Derivation by a 0 , a Xx y and b x gives the following linear equation system: 
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Equation 22 

The system of equations is solvable for N>2. The matrix is only 
dependent on the positions and needs to be inverted only once per image sequence to 
5 solve the system. The modulation depth is then obtained from a Uy and b ix y with 

Equation 5, 

The method can also be used in a comparable manner when the 
projected structure is not sinusoidal by approximating with another function. 

10 Minimizing the bleaching of dyes in fluorescence applications 

In order to minimize the effect of bleaching of dyes, the specimen to 
be investigated is ideally illuminated only during the camera exposure time. The 
influence of a continuous exposure during the recording of, e.g., three individual 

1 5 phase images for generating an optical section increases as the selected exposure 

time decreases compared to the data transfer time (from the camera to the PC) and as 
the exposure intensity increases. Typical data transfer times for digital CCD 
cameras are currently in the range of about 10 ms to 100 ms with a resolution of 
about 512x512 pixel/image. With exposure times in this range, the specimen is 

20 already significantly damaged in some cases without making use of the fluorescence 

emission during the data transfer. For the most careful possible handling of the 
specimen, it is desirable to integrate a fast illumination-side shutter. 

A simple solution consists in the use of electromagnetic "Uniblitz" 
shutters manufactured by Vincent Associates, 1255 University Avenue, Rochester, 

25 NY 14607. An alternative solution consists in the arrangement shown in Figure 5. 

The shutter shown in Figure 5 comprises a plane mirror (22) whose angle relative to 
the optical axis can be adjusted, e.g., by means of a scanner, followed by a prism 
(21) and a stationary mirror (23). An "open" position or through-position (beam 
path with solid lines) and a "closed" or shutter position (beam path in dashed lines) 

30 can be realized by means of this combination. The possible switching times depend 

on the scanner that is used and on the size of the plane mirror. Typical switching 
times for mirror diameters of approximately 25 mm are about 1 ms. 
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Algorithm for minimizing artifacts by bleaching of dyes in 
fluorescence applications 

In order to compensate for bleaching of dyes, the following sequence 
can be used for recording the phase images with a linear approach: 



x,y v~/>- x t y\ ~ 



In 



■'■■'It MtWt ^ m 



1 0 The order of the above sequence represents the time sequence of the acquisition of 

the phase images. From the respective three phase images 
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and 



(2n\ 



V J 



V * J 



I 3 



the optical sections, for example, are to be calculated using Equation 10 and the 
mean value is formed. 



20 



sec tioned„ 



sectioned 



0 >T'T'" 



sec tioned 



^33 



Equation 24 



The formation of artifacts in the calculated optical section image is accordingly 
minimized in a linear approximation of the bleaching process. 



25 



Grating detection, module detection 



For purposes of simple mechanical switching from conventional 
microscope recording to structured illumination, and vice versa, it is advantageous to 
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combine a free position with a field diaphragm with the grating. This is shown in 
Figure 8. 

Since the depth discrimination in structured illumination depends, 
among other things, on the effective grating frequency, which is determined in turn 
5 by the illumination-side total magnification (tube lens + condenser or objective), it is 

advantageous to provide a simple possibility for exchanging the grating used for 
projection. 

To prevent errors in case of incorrectly inserted gratings, it is 
advantageous, for example, to code the mechanically mounted gratings. The coding 
10 shown in Figure 12 can be used for this purpose. 

While the foregoing description and drawings represent the present 
invention, it will be obvious to those skilled in the art that various changes may be 
made therein without departing from the true spirit and scope of the present 
invention. 
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ABSTRACT 



A method and arrangement are disclosed for increasing the depth contrast in 
microscope imaging. The method and implementation described can be designated as 
structured illumination for generating quasi-confocal optical sections. In implementing the 
method, a grating structure located in the field diaphragm plane of a microscope, the object 
plane and the TV intennediate image plane of a microscope are arranged confocally. The 
term "confocally" refers to the fact that the grating, object and the intermediate image plane 
are positioned on optically conjugate planes. By this arrangement, the grating structure is 
projected in the object plane of the microscope and the object which is structured in this way 
is imaged in the TV intermediate image plane of the microscope by the optical system 
following it. Optical sections are generated by calculating the modulation depth of the 
structured object. Three-dimensional acquisition of the object is achieved in that the object is 
imaged in a plurality of focus planes at right angles to the direction of observation and is 
detected using an array detector (e.g., CCD camera). The method and implementation of 
structured illumination described herein can primarily be used in reflection microscopy and 
fluorescence microscopy. In principle, the method can be applied for all linear interactions 
between light and matter. The use of the method is likewise not limited to the field of 
microscopy. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows a simplified optical schematic of structured 
illumination. The image-forming beam path is shown. A one-dimensional 
transmission grating (3) located in a focus plane of the optical arrangement shown in 
the drawing is illuminated by a light source (1) and collector optics (2) succeeding 
the latter. The transmission grating can be moved in axial direction (parallel to the 
optical axis of the system). Possible movements are indicated by the arrow shown in 
the drawing. The grating is succeeded in the direction of light by a plane-parallel 
glass plate (4). The angle of the plane-parallel plate with respect to the optical axis 
can be adjusted in a defined manner. The structure is imaged in the specimen plane 
(7) by the succeeding illumination-side lenses 1 (5 and 6) (tube lens and objective or 
condenser). The possible movements of the lens (5) (tube lens) indicated by an 
arrow can be used as an alternative solution to the movement of the grating (3), since 
an axial displacement of the tube lens substantially equals an axial displacement of 
the focus plane. 

Light proceeding from the specimen is imaged again through a pair of 
lenses (8 and 9) (objective and tube lens) in the focus plane (10) following the latter. 

Figure 2 schematically shows the longitudinal chromatic aberrations 
which occur, in general, in the optical arrangement from Figure 1. The image- 
forming beam paths for two different wavelengths are shown. Due to the 
longitudinal chromatic aberration of the lens combinations, the focus planes for 
different wavelengths extend axially at different locations. This longitudinal 
chromatic aberration can be corrected by an axial displacement of the grating (or of 
the tube lens (5)). This is indicated by the grating positions (3a) and (3b). Further, 
the same axial location in the specimen is not sharply imaged for both wavelengths. 
This is shown by the focus planes (7a) and (7b). The difference in the position of the 
focus planes (7a) and (7b) leads, e.g., with multifluorescence, to an erroneous 
acquisition of the object. The chromatic shift of the focus planes in the object can be 
taken into account by a software correction of the axial positions. 

Figure 3 shows a construction of the optical arrangement from Figure 
1 or Figure 2 for incident fluorescence detection in microscopy. A structure 3 (sine 



1 By "lenses" is meant, in general, also complex lens systems. 



grating or binary transmission grating) is illuminated by a lamp (1) and succeeding 
lens (2). The position of the structure can be varied axially, e.g., by a motor-driven 
eccentric movement (see Figure 7). For control purposes, the motor of the eccentric 
is connected to the control electronics by a cable. A plane-parallel transparent glass 
plate (4) (e.g., made from BK7 or fused silica) succeeding the latter can be offset in 
different angular positions relative to the optical axis, e.g., by means of a scanner 
(Figure 7). For control purposes, the scanner is connected to the electronics box by a 
cable. The spatial phase of the structure can be varied (Figure 5) by the parallel 
offset of the plate. The succeeding (illumination-side) tube lens (5) images the 
structure in the infinity space of the ICS optics of the microscope. The structure is 
imaged in the specimen plane through the excitation filter (7), the dichroic beam 
splitter (1 1) and the objective (10). The preparation or specimen (9) is located on 
the vertically adjustable object stage (8). The height of the stage (8) can be adjusted 
in the direction indicated by the arrow by the drive mechanism (6). For this purpose, 
the drive mechanism is connected to the electronics control unit (15) by a cable. The 
fluorescent light emitted by the specimen is separated from the excitation light by 
the dichroic color splitter and the emission filter (12) succeeding the latter. The 
excitation filter, dichroic beam splitter and emission filter are grouped together (18). 
Finally, the structured fluorescence emission is focused on the intermediate image 
plane of the camera by a tube lens (13). The CCD chip of the camera (14) detects 
the fluorescence emission which is structured in this way. The camera data are 
transferred to the PC (16). The axial position of the structure (grating) and the angle 
of the plane-parallel plate relative to the optical axis can be adjusted by the PC via 
the electronic interface with the control electronics. For this purpose, the control 
electronics are connected to a scanner, e.g., a galvanometer scanner (e.g., M2T, 
manufactured by General Scanning, 500 Arsenal Street, Watertown, MA 02472). 
Finally, the spatial phase of the grating can accordingly be adjusted (Figure 6). 
Further, the axial position of the grating is adjustable. In this way, longitudinal 
chromatic aberrations can be minimized (Figure 2). This enables the spatial-phase- 
dependent acquisition and calculation of the structured images for calculating an 
optical section image under different chromatic conditions. The result of the 
calculation is displayed on the monitor 17. In order to measure the illumination 
intensity, a small proportion of the excitation light is coupled out, e.g., via a plane- 



parallel plate (20), and focused on a photodiode through a lens. The voltage of the 
photodiode is supplied to the control electronics via a cable connection. After 
analog-digital conversion, the digital value is used for correcting instabilities of the 
light source. 

Possible implementations of the axial displacement of the grating and 
adjustment of the angle of the plane-parallel plate are shown in Figure 7. 

Figure 4 shows a corresponding arrangement for a reflecting 
specimen. For this purpose, the component group (18) shown in Figure 3 is replaced 
by a 50:50 beam splitter (21) (e.g., a semitransparent mirror). 

Figure 5 shows an expanded system with an electronic shutter for 
controlling the exposure cycles for camera detection. The shutter shown here 
comprises a plane mirror (22) whose angle relative to the optical axis can be 
adjusted, e.g., by a galvanometer scanner, followed by a prism (21), and a stationary 
mirror (23). Through the combination of the plane mirror whose angle relative to 
the optical axis is adjustable, the prism and the fixed plane mirror, an "open" 
position (beam path indicated by solid lines) and a "closed" position of the optical 
arrangement (beam path in dashed lines) can be realized. 

The arrangement makes it possible to switch the light source on and 
off rapidly. Realistic switching times are in the range of about 1 ms to 10 ms. The 
minimum shutter times are limited by the response times of the scanner with typical 
mirror diameters of about 20 mm. 

The essential advantage in this case consists in the fast switching time 
of the arrangement and the negligible wear of the described arrangement compared 
to normal wear. 

Figure 6 is a schematic diagram showing the adjustment of the spatial 
phase of the grating. The square grating 1 shown in the drawing is imaged via a 
plane-parallel glass plate 2. The image of the grating 3 can be displaced depending 
on the angle of the plate. This makes it possible to adjust the phase of the grating 
without mechanical movement thereof. 

Figure 7a shows a side view of the mimic for adjusting the axial 
position of the grating and the adjustment of the angle of the plane-parallel plate. 
The axial position (height) of the structure (2) can be adjusted by means of a motor 
(4)-controlled eccentric (3). A galvanometer scanner (5) makes it possible to adjust 



the angle of the plane-parallel plate (1) by a rotating movement. The same 
arrangement is shown in a top view in Figure 4b. 

Finally, Figure 8 shows a construction of the overall module in which 
two diaphragm apertures are provided. The aperture designated as position 1 is a 
structure-free optical passage with a variable iris diaphragm (field diaphragm of the 
microscope). Position 2 contains the optical arrangement, which was described in 
detail above, comprising structure and plane-parallel plate in the optical beam path. 
The arrangement shown in this Figure makes it possible to change, for example, 
from conventional incident light microscopy to structured illumination in a simple 
manner. For this purpose, the entire module shown here must be moved 
perpendicular to the optical axis into the desired position. 

Figure 9 shows a possibility for correcting the longitudinal chromatic 
aberration in the LFB plane. For this purpose, two optical glass wedges (1) (2) are 
used. One of the two wedges can be displaced, e.g., by a linear motor, in the 
direction indicated by the arrow. The arrangement shown in the drawing allows the 
optical path length to be varied. 

In Figure 10, the changing of the optical path length for correcting the 
longitudinal chromatic error by means of optical wedges (Figure 9) is integrated in a 
construction of the optical system. This Figure corresponds to Figure 3 with the 
exception of the wedge arrangement (21). 

Figure 1 1 is a schematic view of the microscope system and the 
electronic components. 

Figure 12 shows the carrier of the grating from Figure 7 (2) in an 
embodiment with coding. The five electronic contacts shown in the drawing are 
used by the conducting pins, also shown, to realize the desired binary information 
for automatic detection of the implemented grating carriers. Coded gratings are 
useful, particularly when gratings are exchanged mechanically, so as to prevent user 
errors due, for example, to incorrectly adjusted spatial phases for calculating section 
images by the automatic detection. 

The automatic detection of a mechanical module is implemented in 
Figure 13. A photosensitive receiver can be covered or opened by the relative 
movement, indicated by the arrow, between the two interfaces or boundary surfaces. 
Automatic position detection can be realized in this way. 



[1.2] DESCRIPTION OF THE METHOD AND PRFERRED EMBODIMENTS 

The principle of structured illumination will be described in the 
following. A structure is located in the plane of the field diaphragm (LFB) of a 
microscope. In the following, the structure is a one-dimensional or two-dimensional 
periodic structure. For example, the one-dimensional or two-dimensional period 
structure (grating) can be a transmission grating or reflection grating fabricated by 
lithography. The arrangement of the grating can be binary (square grating) or 
sinusoidal (sine grating). The grating is imaged in the object plane by suitable 
optical elements (illumination-side tube lens and, e.g., objective). A reflecting or 
fluorescing object, for example, is located in the object plane. The lateral structure 
of the illumination intensity, with linear interaction, results in a structured object. 
The structured object is imaged, in turn, via the objective and the TV-side tube lens, 
in the succeeding intermediate image plane and is registered by an array detector (see 
Figure). The sensor surface of the detector is situated in the intermediate image 
plane (TV output at the microscope). 

The increase in depth contrast in a microscope recording with 
structured illumination is ultimately attributable to the fact that only the mean value 
(or DC component) in conventional microscope imaging does not disappear when 
defocusing. However, when an object with a structure whose spatial frequency (or 
frequencies) is (or are) not equal to zero is illuminated and only the proportion of 
image information that was imaged in a structured manner is subsequently evaluated, 
the DC component (zero frequency) can be numerically removed. This increases the 
depth contrast. The structure of the illumination "marks" the portion of the object 
located in the focus of the optical system. 

Essentially the modulation depth (AC component) of the structured 
object must be determined by pixel in order to calculate the optical section. In the 
simplest case of a one-dimensional periodic sinusoidal structure, with a linear 
interaction, the image-side intensity / = I{x,y,<p) with spatial variables x,y and the 
parameter for the spatial phase angle of the periodic structure <p is given by 



I(x,y,(p) = a 0xv +m x y sm(k x -x + cp) 



Equation 1 



where the locally dependent modulation depth m xy =:m(x, y). 

On the one hand, the modulation depth is dependent on the grating 
frequency, the imaging optics and the object being viewed. 

Using addition theorems, this can be written as 

KAV) = a or,y+ a u,v -sin(0>) + b Xxy -cos(cp) Equation 2 

where (p is the phase angle of the sinusoidal structured illumination, cio x ,y is the DC 
component (= mean value) and 

a lx y =m x y cos(k x x) Equation 3 

b U y = m x y s\n{k x x) Equation 4 

so that it immediately follows for the local modulation depth of the sine frequency: 



m Xf}} = ^ aix,y +b{Xi ^- oc ^ja ]x / +b lx y 2 . Equation 5a 

Cl 0x,y 



An optical section image is precisely the modulation depth multiplied by the image 
intensity (= DC component) 



h,cnoned Ky = » A0 ,'= ™V = ^J^^ Equation 5b 

In addition, the phase angle can be calculated 



tan( 9 ) = ^ 



Equation 6 



The coefficients a /XJ ,and b Ix ^ which are dependent on the modulation 
frequency, and the locally dependent mean value can initially be written as follows 
by changing the phase of the discrete Fourier expansion: 

Equation 7 



'Ov,y 



l £! . I 2^ 



7 = iSM" 



n=0 



NJ 



N 



»=0 



( 2%~\ 




( 2k ^ 


n — 




> n — 


I Nj 







Equation 8 



2?y f 2n\ ( 2tt^ 



Equation 9 



where N is the quantity of phase steps. 

By inserting Equations 7 - 9 in Equation 5, the local modulation 

depth m XJ ,canbe calculated. 

For three or four phase steps, the formulas are simplified for the last 

optical sections sought: 



I sectioned xy ^x,y 



3 



( 4;r 



l x,y 



- I 



*,y 



Equation 10 



for three phase steps (0, 2tt/3, 4tc/3), and 



1 

/ . _ =m ' = - 
sectioned x,y 2 \ 

xy 1 



(0) - / («•) 
y x,y 



*,H 2 J x,yy 2 

Equation 1 1 



for four phase steps (0, n/2 9 n, 3rc/2). 
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In order to calculate the DC component, the "mean value" must be 
calculated over the phase images (Equation 7). For three phase angles: 



/ x.y = a. 



(In 

^(0) + / ( ,| T | + ^.v 



4ll 

IT 



and, in a corresponding manner, for four phase angles: 



Equation 12a 



3k ^ 



Equation 12b 



steps: 



The simplest case is the calculation of the mean value with two phase 



I, y 



^(/,,(0) + / v ») 



Equation 12c 



Because of the displacement set for the trigonometric functions, the 
modulation frequencies are eliminated in the calculation with the help of the last 
equations 12c. This simple algorithm is especially advantageous also when using a 
square grating (see below) because all higher harmonics of the square grating are 
suppressed. For N= 2", where n is a natural number, this is always the case. In 
general, when more than two phase steps are used, this is not the case. For example, 
with three phase steps (Equation 12a) the third harmonic leads to stripe artifacts 
when the mean value is formed. 

The DC component calculated generally by Equation 7 and Equation 
12 corresponds to conventional recording in the microscope. The depth contrast is 
not increased in this case. The calculation of the brightfield information with 
Equation 7 (12) is particularly applicable in an advantageous manner, since this 
processing makes it possible to switch from optical section to conventional 
recording without having to mechanically remove the structure from the beam path. 

When both the DC component and an optical section (AC 
component) are calculated, the out-of-focus information can also be determined 
(after suitable calibration) by subtraction (DC-AC). 



When using a square grating, the following series expansion can be 
used for illustration: 



w M2« + i> fi , Sm 



k x (2i + l) 



x J J 



Equation 13 



(2i + l) 

The modulation depth of the grating frequency when focusing, e.g., 
on a mirror as specimen, can be described by the modulation transfer function [T. 
Wilson]. 



m, = 



71 



acos ! 



kX \ kX 



1- 



IN A) 2NA\ {2NA 



kX 



Equation 14 



with incoherent illumination and incident illumination, where 

NA = numerical aperture 

X = wavelength 

k = spatial frequency. 

The formula above is applicable for circular pupils. In defocusing, a wave 
aberration, as complex phase factor, must be multiplied by the pupil function. 

Referring to Equation 10, it will be seen that the three-phase formula 
implicitly suppresses the third harmonic of the Fourier expansion of a square grating 
(Equation 13, i-1). Not until the fifth harmonic (i=2) does an interference term and, 
therefore, a residual modulation (artifact) result. When the one-dimensional grating 
is so dimensioned that the fifth harmonic can not be transmitted through the optical 
lowpass, i.e., is selected so as to be greater than the optical limit frequency 



k - ziyA Equation 15 

"-limit ^ ' 

the artifacts can be completely suppressed by higher harmonics. This is not the case 
when Equation 1 1 is applied, i.e., when using four phase steps for calculating the 
(local) modulation depth. Therefore, Equation 1 1 can be used in connection with 
square gratings only to a limited extent, e.g., in case even the third harmonic in the 
Fourier expansion is not transmitted through the optics. It is advantageous with 
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respect to Equation 1 1 , under this additional pre-condition, that the second harmonic 
of the grating can be implicitly suppressed by the algorithm, 

[1.3] Technical Implementation 

Technical implementations are shown in Figures 3 and 4 and further 
details on the implementations are shown in Figures 5 to 8. 

The following details of the implementations which are to be applied 
in the imaging systems for application in fluorescence microscopy (Figure 3) and 
reflection microscopy (Figure 4) will once again be presented in this connection. 
The relevant implementations are: 

Phase adjustment by means of a plane-parallel plate (plane- 
plate micrometer) 

The (motor-actuated) movement of the grating in axial 
direction for correcting longitudinal chromatic errors (focus correction) 

Correction of instability in the light source through the 
measurement of the light intensity by means of a photodetector (typically a 
photodiode) and subsequent numerical correction by scaling the acquired individual 
phase images 

Correction, e.g., of bleaching of dyes in fluorescence 
microscopy by means of an optimized averaging process 

Implementation of a fast shutter by means of scanners 
corresponding to Figure 5 

Correction, e.g., of statistical changes in the spatial phase of 
the grating by solving a system of equations 

Implementation of a system for the use of a two-dimensional 
grating using two orthogonal plane-plate micrometers. 

[1.31] Phase displacement by means of a plane-parallel plate 

The adjustment of the phase position of the grating can be 
advantageously realized by means of parallel displacement (plane-plate micrometer). 
The principle is shown in Figure 6. A plane-parallel glass plate (e.g., made of BK7) 
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is arranged after the grating and, e.g., well-defined angles are adjusted by means of a 
galvanometer scanner. The use of a scanner offers the advantage in particular of a 
precise and fast adjustment of the angle. Typical adjusting angles in the range of 
about +/- 15° with an adjusting accuracy of <50 prad and an adjusting time of less 
than 5 ms are technically possible. 

The data can be recorded synchronous with the detector in two ways: 
by a continuous movement of the plane-parallel plate and 
synchronization with the camera 

by a stepwise, sequential approach to the scan position 
Sequential positioning offers the advantage of a higher modulation 
depth (= contrast) of the structured object and independence of the contrast from the 
ratio of exposure time to readout time of the camera. 

In order to optimize the excitation-side transmission on the one hand 
and to minimize multiple reflections at the plane-parallel plate on the other hand, it 
is advantageous to minimize the reflections at all optical components, particularly 
the structure (or the substrate) and the plane-parallel plate, by antireflection coatings. 
Further, it is advantageous to insert the plane-parallel plate into the optical beam 
path at a small angle (not equal to 0°). Interfering reflections can be eliminated in 
this way. 

[1.3.2] Focus correction, compensation of longitudinal chromatic aberration 

Although the imaging, for example, of fluorescing objects and 
reflecting objects, is identical in some respects, the differences between excitation 
wavelength and emission wavelength (Stokes shift), e.g., in fluorescing objects, 
present several essential points to be considered. 

Since, as a rule, chromatically corrected objectives also have a 
longitudinal chromatic aberration, the grating, object and TV intermediate image 
plane are each located in the optically conjugate planes only for one 
excitation/emission wavelength. With a change in the spectral ratios (filter change) 
or when the objectives are exchanged, the focus position of the object, grating and 
camera that has already been adjusted is no longer given. This leads to a reduction 
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in contrast in the structured image and, with larger longitudinal chromatic 
aberrations, to stripe artifacts in the calculated optical section image. 

The focus adjustment of the object on the intermediate image plane of 
the camera can be realized by refocusing. However, it is also necessary to correct 
the grating position relative to the object plane. The advantageous use of a manual 
or motor-operated adjusting possibility for focus adjustment of the grating on the 
object is shown in the present patent application. This can be implemented in two 
different ways using ICS optics: 

a) Displacement of the tube lens in Figure 1 . 

b) Displacement of the grating position in Figure 2. 
Even when the wavelengths for excitation and detection are 

substantially identical (reflection microscopy), an axial movement of the structure or 
illumination-side tube lens may also be necessary for correction due to chromatic 
aberrations. In particular, this is the case when changing from the visible spectral 
range to UV, for example. 

The grating position can be adapted, for example, by an eccentric 
movement (Figures 7a, 7b). Given sufficient space (or a sufficiently small necessary 
focus correction), it is possible to use an optical wedge and, therefore, to change the 
optical path length (Figures 9, 10). 

[1 3.3] Correction of time-dependent intensity instability of the light source 

A change over time in the intensity of the light source during the 
detection of the phase images, generally N>=3 phase images, leads to stripe artifacts 
in the calculated optical section image, since a light source that is constant over time 
is assumed in the derivation of the formulas given above. The following steps can 
be taken to eliminate this influence of time-dependent changes in illumination which 
interferes with image quality. In a first step, a measurement value for the 
illumination intensity (shown in Figures 3, 4, 5) is obtained for the intensity of the 
light source, e.g., by imaging on a light-sensitive receiver, e.g., a commercially 
available photodiode. The time interval for the integration must be matched to the 
time interval for the exposure of the CCD chip of the camera. The voltage signal of 
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the photodiode circuit is subsequently digitized by means of an A-D converter. 
After measurement of the N individual images with the spatial phases 



Inn 

they can be scaled, e.g., as follows: 



Equation 16 



j (0 )= /y > ^ Equation 17 



with the digitized photodiode signal 



N-\ 



T PD S l8n at(<Pn) 



PD -=JirO Equation 18 

rLJ signal N n 



averaged over the image phases, and the correction factor 



C(a> \ - = PD Signal (ffn) Equation 19 

PD 

r ^Signal 



[13 A] Phase instability, phase adjustment speed 

hi order to minimize phase instability, it is advantageous to use a 
temperature-stabilized galvanometer scanner (e.g., M2T, manufactured by General 
Scanning). A scanner of this type also enables a short phase adjustment time of a 
few milliseconds, hi order to optimize phase stability, the scanner is controlled 
electronically by a closed control loop. 

In addition, an SW correction of phase instabilities can be carried out 
by measuring the phases and subsequently approximating with a sine function 
according to the method described in the following. 

While an exact displacement of the projected structure by an Nth of 
the period is assumed in Equations 7-9, the modulation depth can also be 
determined when the intensity is recorded in three or more virtually optional 
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positions. First, the phase angle (p in Equation 1 must be measured for all positions 
of the structure. This can be carried out in different ways: 

1 . By measurement at the mechanical actuating element 
responsible for the displacement of the structure (e.g., the scanner) 

2. By determining from the recorded image, e.g., by Fourier 
transformation or by approximation of the recorded intensity values with a sine 
function 

3. Determining as in 2) above, but with a sensor which directly 
records the projected structure. This can also be achieved in that an optical element 
which images the structure directly on a partial area of the detector that is also used 
for recording the intensities of the specimen is arranged in the beam path. 

When determining the displacement of the structure, it is sufficient to 
determine the displacement relative to the first position. 

To determine the modulation depth, N intensity values which were 
recorded at N positions of the structure are considered and the index n is given in 
Equation 2 for the nth position: 

«P U ) - <W + ' Sin ^> ) + *i ' cos ^ « > Equation 20 

The measured intensities M vj O„) can deviate from the expected 
intensities h, y ((Pn)> The method of least error squares can be used for approximation 



Equation 21 



Derivation by a 0jcJ ,, a ljej , and b [Xy) , gives the following linear equation system: 
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Equation 22 

The system of equations is solvable forN>2. The matrix is only 
dependent on the positions and needs to be inverted only once per image sequence to 
solve the system. The modulation depth is then obtained from a lx y and b U y with 

Equation 5. 

The method can also be used in a comparable manner when the 
projected structure is not sinusoidal by approximating with another function. 

[1.3.5] Minimizing the bleaching of dyes in fluorescence applications 

In order to minimize the effect of bleaching of dyes, the specimen to 
be investigated is ideally illuminated only during the camera exposure time. The 
influence of a continuous exposure during the recording of, e.g., three individual 
phase images for generating an optical section increases as the selected exposure 
time decreases compared to the data transfer time (from the camera to the PC) and as 
the exposure intensity increases. Typical data transfer times for digital CCD 
cameras are currently in the range of about 10 ms to 100 ms with a resolution of 
about 512x512 pixel/image. With exposure times in this range, the specimen is 
already significantly damaged in some cases without making use of the fluorescence 
emission during the data transfer. For the most careful possible handling of the 
specimen, it is desirable to integrate a fast illumination-side shutter. 

A simple solution consists in the use of electromagnetic "Uniblitz" 
shutters manufactured by Vincent Associates, 1255 University Avenue, Rochester, 
NY 14607. An alternative solution consists in the arrangement shown in Figure 5. 
The shutter shown in Figure 5 comprises a plane mirror (22) whose angle relative to 
the optical axis can be adjusted, e.g., by means of a scanner, followed by a prism 
(21) and a stationary mirror (23). An "open" position or through-position (beam 
path with solid lines) and a "closed" or shutter position (beam path in dashed lines) 
can be realized by means of this combination. The possible switching times depend 
on the scanner that is used and on the size of the plane mirror. Typical switching 
times for mirror diameters of approximately 25 mm are about 1 ms. 
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[1 3.6] Algorithm for minimizing artifacts by bleaching of dyes in 

fluorescence applications 

In order to compensate for bleaching of dyes, the following sequence 
can be used for recording the phase images with a linear approach: 

^XO),4,(f}/.4f},4^)^f),,,,<o> 

The order of the above sequence represents the time sequence of the acquisition of 
the phase images. From the respective three phase images 



/,,v(0),A,,[ 



2n^ 



J'Mt 



and 



H 3 / X) 



v 



,/,, v (0), 



the optical sections, for example, are to be calculated using Equation 10 and the 
mean value is formed. 



^sec tioned n . 



^> ^ 9 ^ | + 1 sectioned A 1 V ' b 



3 3 J seciionea xy I 3 3 



j = . i . i ^ — £ Equation 24 

sectioned^ ^ 

The formation of artifacts in the calculated optical section image is accordingly 
minimized in a linear approximation of the bleaching process. 

[1.4] Grating detection, module detection 



For purposes of simple mechanical switching from conventional 
microscope recording to structured illumination, and vice versa, it is advantageous to 
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combine a free position with a field diaphragm with the grating. This is shown in 
Figure 8. 

Since the depth discrimination in structured illumination depends, 
among other things, on the effective grating frequency, which is determined in turn 
5 by the illumination-side total magnification (tube lens + condenser or objective), it is 

advantageous to provide a simple possibility for exchanging the grating used for 
projection. 

To prevent errors in case of incorrectly inserted gratings, it is 
advantageous, for example, to code the mechanically mounted gratings. The coding 
1 0 shown in Figure 12 can be used for this purpose. 

While the foregoing description and draw ings represent the 
present invention, it will be obvious to t hose skilled in the art that various 
changes may be made therein without de parting from the true spirit and scope 
of the present invention. 
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MARKED UP/BOLDED SUBSTITUTE SPECIFICATION & ABSTRACT 

Abstract 

[The invention is directed to al A method and arrangement are 
disclosed for increasing the depth contrast in microscope imaging. The method and 
implementation described can be designated as structured illumination for generating 
quasi-confocal optical sections [[1], [2], [3], [4], [5], [6], [7], 18], [9], [10|1. In 

implementing the method, a grating structure located in the field diaphragm plane of 
a microscope, the object plane and the TV intermediate image plane of a microscope 
are arranged confocally [[4], [5], [6], [7], [9]]. The term "confocally" refers to the 
fact that the grating, object and the intermediate image plane are positioned on 
optically conjugate planes. By [means of] this arrangement, the grating structure is 
projected in the object plane of the microscope and the object which is structured in 
this way is imaged in the TV intermediate image plane of the microscope by the 
optical system following it. Optical sections are generated by calculating the 
modulation depth of the structured object. Three-dimensional acquisition of the 
object is achieved in that the object is imaged in a plurality of focus planes at right 
angles to the direction of observation and is detected using an array detector (e.g., 
CCD camera). The method and implementation of structured illumination described 
herein can primarily be used in reflection microscopy and fluorescence microscopy. 
In principle, the method can be applied for all linear interactions between light and 
matter. The use of the method is likewise not limited to the field of microscopy. 
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